Editorial: Perceptual issues surrounding the electroacoustic listening experience

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link

Hearing triangles: perceptual clarity, opacity, and symmetry of spectrotemporal sound shapes

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.In electroacoustic music, the spectromorphological approach commonly employs analogies to non-sonic phenomena like shapes, gestures, or textures. In acoustical terms, sound shapes can concern simple geometries on the spectrotemporal plane, for instance, a triangle that widens in frequency over time. To test the auditory relevance of such triangular sound shapes, two psychoacoustic experiments assessed if and how these shapes are perceived. Triangular sound-shape stimuli, created through granular synthesis, varied across the factors grain density, frequency and amplitude scales, and widening vs. narrowing orientations. The perceptual investigation focused on three auditory qualities, derived in analogy to the visual description of a triangle: the clarity of the triangular outline, the opacity of the area enclosed by the outline, and the symmetry along the vertical dimension. These morphological qualities seemed to capture distinct perceptual aspects, each linked to different acoustical factors. Clarity of shape was conveyed even for sparse grain densities, while also exhibiting a perceptual bias for widening orientations. Opacity varied as a function of grain texture, whereas symmetry strongly depended on frequency and amplitude scales. The perception of sound shapes could relate to common perceptual cross-modal correspondences and share the same principles of perceptual grouping with vision

Timbre blending of wind instruments: acoustics and perception

The acoustical and perceptual factors involved in timbre blending between orchestral wind instruments are investi- gated based on a pitch-invariant acoustical description of wind instruments. This description involves the estimation of spectral envelopes and identification of prominent spectral maxima or ‘formants’. A possible perceptual relevance for these formants is tested in two experiments employing differ- ent behavioral tasks. Relative frequency location and mag- nitude differences between formants can be shown to bear a pitch-invariant perceptual relevance to blend for several in- struments, with these findings contributing to a perceptual theory of orchestration

A spectral-envelope synthesis model to study perceptual blend between wind instruments

Wind instrument sounds can be shown to be characterized by pitch-invariant spectral maxima or formants. An acoustical signal-analysis approach is pursued to obtain spectral-envelope descriptions that reveal these pitch- invariant spectral traits. Spectral envelopes are estimated empirically by applying a curve-fitting procedure to a composite distribution of partial-tone frequencies and amplitudes obtained across an instrument’s pitch range. A source-filter synthesis model is designed based on two independent formant filters with their frequency responses matched to the spectral envelope estimates. This is then used in perceptual experiments in which parameter variations of the synthesis filter are manipulated systematically to investigate their contribution to the degree of per- ceived blend between the synthesized sound and a recorded instrument sound. The perceptual relevance is assessed through two tasks in which participants either produce the best attainable blend by directly controlling synthesis parameters or rate the degree of blend for 5 parameter presets. Behavioral data from both experiments suggest the utility of this formant-based model for correlating pitch-invariant acoustical description with perceptual relevance, as both formant frequency and magnitude appear to affect perceived blend

Blending between bassoon and horn players: an analysis of timbral adjustments during musical performance

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Achieving a blended timbre between two instruments is a common aim of orchestration. It relates to the auditory fusion of simultaneous sounds and can be linked to several acoustic factors (e.g., temporal synchrony, harmonicity, spectral relationships). Previous research has left unanswered if and how musicians control these factors during performance to achieve blend. For instance, timbral adjustments could be oriented towards the leading performer. In order to study such adjustments, pairs of one bassoon and one horn player participated in a performance experiment, which involved several musical and acoustical factors. Performances were evaluated through acoustic measures and behavioral ratings, investigating differences across performer roles as leaders or followers, unison or non-unison intervals, and earlier or later segments of performances. In addition, the acoustical influence of performance room and communication impairment were also investigated. Role assignments affected spectral adjustments in that musicians acting as followers adjusted toward a `darker' timbre, i.e., realized by reducing the frequencies of the main formant or spectral centroid. Notably, these adjustments occurred together with slight reductions in sound level, although this was more apparent for horn than bassoon players. Furthermore, coordination seemed more critical in unison performances and also improved over the course of a performance. These findings compare to similar dependencies found concerning how performers coordinate their timing and suggest that performer roles also determine the nature of adjustments necessary to achieve the common aim of a blended timbre

Acoustical correlates of perceptual blend in timbre dyads and triads

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Achieving a blended timbre for particular combinations of instruments, pitches, and articulations is a common aim of orchestration. This involves a set of factors that this study jointly assesses by correlating the perceptual degree of blend with the underlying acoustical characteristics. Perceptual blend ratings from two experiments were considered, with the stimuli consisting of: 1)~dyads of wind instruments at unison and minor-third intervals and at two pitch levels, and 2)~triads of wind and string instruments, including bowed and plucked string excitation. The correlational analysis relied on partial least-squares regression, as this technique is not restricted by the number and collinearity of regressors. The regressors encompassed acoustical descriptors of timbre (spectral, temporal, and spectrotemporal) as well as ones accounting for pitch and articulation. From regressor loadings in principal-components space, the major regressors leading to substantial and orthogonal contributions were identified. The regression models explained around 90\% of the variance in the datasets, which was achievable with less than a third of all regressors considered initially. Blend seemed to be influenced by differences across intervals, pitch, and articulation. Unison intervals yielded more blend than did non-unison intervals, and the presence of plucked strings resulted in clearly lower blend ratings than for sustained instrument combinations. Furthermore, prominent spectral features of instrument combinations influenced perceived blend

When timbre blends musically: perception and acoustics underlying orchestration and performance

Blending or contrasting instrumental timbres are common techniques employed in orchestration. Both bear a direct relevance to the perceptual phenomenon of auditory fusion, which in turn depends on a series of acoustical cues. Whereas some cues relate to musical aspects, such as timing and pitch relationships, instrumentation choices more likely concern the acoustical traits of instrument timbre. Apart from choices made by composers and orchestrators, the success of timbre blending still depends on precise execution by musical performers, which argues for its relevance to musical practice as a whole.This thesis undertakes a comprehensive investigation aiming to situate timbre blend in musical practice, more specifically addressing the perceptual effects and acoustical factors underlying both orchestration and performance practice. Three independent studies investigated the perception of blend as a function of factors related to musical practice, i.e., those derived from musical context and realistic scenarios (e.g., pitch relationships, leadership in performance, room acoustics).The first study establishes generalized spectral descriptions for wind instruments, which allow the identification of prominent features assumed to function as their timbral signatures. Two listening experiments investigate how these features affect blend by varying them in frequency, showing a critical perceptual relevance. The second study considers two other listening experiments, which evaluate perceived blend for instrument combinations in dyads and triads, respectively. Correlational analyses associate the obtained blend measures with a wide set of acoustic measures, showing that blend depends on pitch and temporal relationships as well as the previously identified spectral features. The third study extends the previous ones, addressing factors related to musical performance by investigating the timbral adjustments performers employ in blending with one another, as well as their interactive relationship. Timbral adjustments can be shown to be made towards the musician leading the performance.All studies contribute to a greater understanding of blend as it applies to musical and orchestration practice. Their findings expand previous research and provide possible explanations for discrepancies between hypotheses made in the past. Together, the conclusions drawn allow us to propose a general perceptual theory for timbre blend as it applies to musical practice, which considers the musical material and spectral relationships among instrument timbres as the determining factors.Fusionner ou différencier les timbres instrumentaux sont des techniques d'orchestration courantes. Elles présentent toutes deux un intérêt direct pour le phénomène de fusion auditive, qui dépend d'une série d'indices acoustiques. Alors que certains indices sont liés aux aspects musicaux comme la synchronisation ou les relations de hauteurs perçues, les choix d'instrumentation sont davantage liés aux traits acoustiques du timbre de l'instrument. En plus des choix faits par les compositeurs et les orchestrateurs, le succès de la fusion des timbres tient de la précision de l'exécution des instrumentistes, ce qui renforce encore sa pertinence pour la pratique musicale en général.Cette thèse présente une étude approfondie de la place de la fusion des timbres dans le jeu musical, et s'intéresse plus particulièrement aux effets perceptifs et aux facteurs acoustiques sous-jacents à l'orchestration et à la pratique instrumentale. Trois études indépendantes ont été conduites pour étudier la perception de la fusion en fonction de facteurs liés à la pratique musicale, c'est-à-dire, découlant du contexte musical et de scénarios réalistes comme les relations entre les hauteurs perçues, le leadership pendant le jeu, l'acoustique de la salle.La première étude propose des descriptions spectrales généralisées pour les instruments à vent, ce qui permet l'identification des descripteurs les plus importants pouvant représenter leur signature de timbre. Deux tests d'écoute étudient leur influence sur la fusion en les faisant varier en fréquence, ce qui démontre leur pertinence sur le plan perceptif. La seconde étude est fondée sur deux autres tests d'écoute ayant pour but d'évaluer la fusion perceptive lors de combinaison d'instruments, respectivement présentés en dyade et en triade. Des analyses de corrélation montrent une association entre les mesures obtenues sur la fusion et de nombreuses mesures acoustiques, et montrent que la fusion dépend de la hauteur et des relations temporelles mais également des caractéristiques spectrales identifiées précédemment. La troisième étude complète les précédentes en ce sens qu'elle s'intéresse aux facteurs liés à la performance musicale en étudiant les ajustements de timbre auxquels les musiciens ont recours lorsqu'ils cherchent à fusionner leurs jeux, et comment ces ajustements sont interdépendants. Il est possible de montrer que ces ajustements de timbre sont exécutés en fonction du musicien qui guide la performance.Toutes ces études contribuent à une meilleure compréhension de la fusion, appliquée au jeu musical et à l'orchestration. Les résultats obtenus permettent de compléter les recherches existantes sur le sujet en ce sens qu'ils apportent des explications possibles aux divergences existant entre les diff ́erentes hypothèses formulées par le passé. Finalement, les conclusions de cette thèse permettent d'établir une théorie perceptive générale pour la fusion de timbre en contexte musical, qui pose le matériel musical et les relations spectrales entre timbres instrumentaux comme facteurs déterminants

Ratings of auditory shape for pitch and loudess gestures: Averaged data across triplet contexts

This dataset contains gesture-shape ratings that were averaged across the so-called triplet contexts. These ratings were used in the analyses of variance (ANOVA) reported in the article. Data for pitch and loudness gestures are in separate CSV files.  Each CSV file is formatted in "long format", that is, each row corresponds to one observation (dependent variable) for a specific stimulus condition (combination of independent variables) and one participant. The table columns contain the following information, from left to right: DV, dependent variable: Gesture-shape rating for either pitch or loudness acousFct: Acoustical/physical function or scale underlying the pitch or loudness variation orient: Orientation of gesture, either increasing or decreasing over time durat: Duration, either short or long level: Investigated level range, lower vs. higher for pitch, softer vs. louder for loudness pa: Participant identification number (arbitrary assignment) </ul

Ratings of auditory shape for pitch, loudness, and tempo gestures: Raw data across all triplet contexts

This dataset contains all raw gesture-shape ratings acquired during the experiment, notably, across so-called triplet contexts. Data for pitch, loudness, and tempo gestures are in separate CSV files. Each CSV file is formatted in "long format", that is, each row corresponds to one observation (dependent variable) for a specific stimulus condition (combination of independent variables) and one participant. The table columns contain the following information, from left to right: DV, dependent variable: Gesture-shape rating for either pitch, loudness or tempo acousFct: Acoustical/physical function or scale underlying the pitch, loudness or tempo variation, also denoted as letters A to D context: Triplet context of acoustical functions that was evaluated in the experimental trial, that is, ABC, ABD, ACD, BCD orient: Orientation of gesture, either increasing or decreasing over time durat: Duration, either short or long, not applicable for tempo level: Investigated level range, lower vs. higher for pitch, softer vs. louder for loudness, slower vs. faster for tempo pa: Participant identification number (arbitrary assignment) </ul

Distinguishing between straight and curved sounds: Auditory shape in pitch, loudness, and tempo gestures

Sound-based trajectories or sound gestures draw links to spatiokinetic processes. For instance, a gliding, decreasing pitch conveys an analogous downward motion or fall. Whereas the gesture’s pitch orientation and range convey its meaning and magnitude, respectively, the way in which pitch changes over time can be conceived of as gesture shape, which to date has rarely been studied in isolation. This article reports on an experiment that studied the perception of shape in uni-directional pitch, loudness, and tempo gestures, each assessed for four physical scalings. Gestures could increase or decrease over time and comprised different frequency and sound level ranges, durations, and different scaling contexts. Using a crossmodal-matching task, participants could reliably distinguish between pitch and loudness gestures and relate them to analogous visual line segments. Scalings based on equivalent-rectangular bandwidth (ERB) rate for pitch and raw signal amplitude for loudness were matched closest to a straight line, whereas other scalings led to perceptions of exponential or logarithmic curvatures. The investigated tempo gestures, by contrast, did not yield reliable differences. The reliable, robust perception of gesture shape for pitch and loudness has implications on various sound-design applications, especially those cases that rely on crossmodal mappings, e.g., visual analysis or control interfaces like audio waveforms or spectrograms. Given its perceptual relevance, auditory shape appears to be an integral part of sound gestures, while illustrating how crossmodal correspondences can underpin auditory perception.</p